HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics

To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude.Comment: 10 pages, 8 figures, 5 listings, 1 tabl

HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics

In order to cope with the exponential growth in available data, the efficiency of data analysis and machine learning libraries have recently received increased attention. Although corresponding array-based numerical kernels have been significantly improved, most are limited by the resources available on a single computational node. Consequently, kernels must exploit distributed resources, e.g., distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload via MPI on arbitrarily large high-performance computing systems. It provides both low-level array-based computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take advantage of their available resources, significantly lowering the barrier to distributed data analysis. Compared with applications written in similar frameworks, HeAT achieves speedups of up to two orders of magnitude

HeAT - a Distributed and GPU-accelerated Tensor Framework for Data Analytics

To cope with the rapid growth in available data, theefficiency of data analysis and machine learning libraries has re-cently received increased attention. Although great advancementshave been made in traditional array-based computations, mostare limited by the resources available on a single computationnode. Consequently, novel approaches must be made to exploitdistributed resources, e.g. distributed memory architectures. Tothis end, we introduce HeAT, an array-based numerical pro-gramming framework for large-scale parallel processing withan easy-to-use NumPy-like API. HeAT utilizes PyTorch as anode-local eager execution engine and distributes the workloadon arbitrarily large high-performance computing systems viaMPI. It provides both low-level array computations, as wellasassorted higher-level algorithms. With HeAT, it is possible for aNumPy user to take full advantage of their available resources,significantly lowering the barrier to distributed data analysis.When compared to similar frameworks, HeAT achieves speedupsof up to two orders of magnitude

HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics

To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude

Heat - Helmholtz Analytics Toolkit - v1.2.0

Google Summer of Code 2022; support for PyTorch 1.11; data-intensive signal processing; Parallel writing out to CSV file; more flexibility in memory-distributed binary operations; expanded functionalities in linalg, manipulations modules

helmholtz-analytics/heat: Heat 1.0: Data Parallel Neural Networks, and more

Release Notes Heat v1.0 comes with some major updates: new module nn for data-parallel neural networks Distributed Asynchronous and Selective Optimization (DASO) to accelerate network training on multi-GPU architectures support for complex numbers major documentation overhaul support channel on StackOverflow support PyTorch 1.8 stop supporting Python 3.6 many more updates and bug fixes, check out the CHANGELO

helmholtz-analytics/heat: Heat 1.1.0: distributed slicing/indexing overhaul, dealing with load imbalance, and more

Highlights Slicing/indexing overhaul for a more NumPy-like user experience. Special thanks to Ben Bourgart @ben-bou and the TerrSysMP group for this one. Warning for distributed arrays: breaking change! Indexing one element along the distribution axis now implies the indexed element is communicated to all processes. More flexibility in handling non-load-balanced distributed arrays. More distributed operations, incl. meshgrid . For other details, see the CHANGELOG